Privacy-Preserving SGD on Shuffle Model

In this paper, we consider an exceptional study of differentially private stochastic gradient descent (SGD) algorithms in the convex optimization (SCO). The majority existing literature requires that losses have additional assumptions, such as loss functions with Lipschitz, smooth and strongly convex, uniformly bounded model parameters, or focus on Euclidean (i.e. l 2 d ) setting. However, these restrictive requirements exclude many popular losses, including absolute hinge loss. By loosening restrictions, proposed two SGD without shuffle (in short, DP-SGD-NOS DP-SGD-S) for id="M2"> α , L -Hölder by adding calibrated Laplace noise under no shuffling scheme id="M3"> p -setting id="M4"> ∈ open="[" close="]" 1,2 . We provide privacy guarantees using advanced composition amplification techniques. also analyze convergence bounds DP-SGD-S obtain optimal excess population risks id="M5"> mathvariant="script">O 1 / n + mathvariant="normal">log δ ϵ id="M6"> 4 up to logarithmic factors complexity id="M7"> − It turns out utility bound is superior model, which consistent previous work. addition, achieves id="M8"> computations linearity id="M9"> ≥ There a significant tradeoff between id="M10"> differential model.